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Abstract 


The discipline of assessment has matured to the point where there is general agreement on best practices. However, the field 
has made little progress in developing a theoretical basis — whether for assessment in general or for more specific 
dimensions of assessment as they emerge. Without a generalizable theory to work from, assessment professionals remain 
focused on the details of practice — getting it done — instead of turning their attention to systems thinking in the service of 
improving, revising, growing, or otherwise developing a field that is still far from perfect. In this article, the authors bring 
sociological theory to bear on learning outcomes assessment in order to understand its strengths and challenges from a 
systems point of view. Then, using this theoretical understanding, we propose an alternative method of assessment 
(Assessment 2.0) designed to supplement the assessment work already being done while at the same time avoiding its most 
difficult challenges. Assessment 2.0 is organic because it grows naturally from the professional judgment and experience of 
instructors rather than from the highly structured, linear procedure commonly followed in standard assessment practice. 
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Assessment 2.0: An Organic Supplement to MI 


Standard Assessment Procedure ar 
Using sociological theory 


Eric T. Metzler and Lisa Kurz to understand assessment’s 


While the idea of establishing student learning outcomes and assessing students’ strengths and challenges from 
progress toward achieving them has been a part of U.S. educational practice at a systems point of view, we 
least since the time of John Dewey (Ewell, 2007), the assessment of learning propose an alternative mode 
in higher education as a movement is generally seen as beginning in the 1980s of assessment (Assessment 
(Ewell, 2002, 2009; Weiss, Cosbey, Habel, Hanson, & Larsen, 2002), and 
since then it has grown in significance. The discipline of assessment—if it may 


2.0) designed to supplement 


so be called—has matured to the point where there is general agreement on the assessment work already 


best practices (e.g., Maki, 2010; Suskie, 2009; Walvoord, 2004). However, the being done while at the same 
field has made little progress in developing a theoretical basis—whether for time avoiding its most difficult 
assessment in general or for more specific dimensions of assessment as they challenges. 

emerge. Without a generalizable theory to work from, assessment professionals 

remain focused on the details of practice—getting it done—instead of turning 

their attention to systems thinking in the service of improving, revising, 

growing, or otherwise developing a field that is still far from perfect. In this 

article, we bring sociological theory to bear on learning outcomes assessment in 

order to understand its strengths and challenges from a systems point of view. 

‘Then, using this theoretical understanding, we propose an alternative method 

of assessment (Assessment 2.0) designed to supplement the assessment work 

already being done while at the same time avoiding its most difficult challenges. 

As we shall discuss below, our proposal for Assessment 2.0 is organic because it 

grows naturally from the professional judgment and experience of instructors 

rather than from the highly structured, linear procedure commonly followed in 

standard assessment practice. 


Standard assessment practice follows a common process: articulating goals 
and learning outcomes, aligning the curriculum, identifying measures and 
instruments, collecting and analyzing data, and using the results to improve 
student learning (Pratto, 1996; Roscoe, 2017; Hatfield, 2009; Ewell, 2002; 
Martell & Calderon, 2005; Weiss et al., 2002). While this process has many 
strengths, including its logical, linear design, its visual depiction of whole 
curricula (tabular curriculum maps), and its focus on measurement of specific 
student learning, it can fall short when one considers the ultimate purpose of 
learning outcomes assessment. As assessment experts have pointed out, most 
institutions begin and execute the process up through the data collection step; 
fewer have been successful in using the data to make programmatic (as opposed 
to course-level) decisions that substantively improve student learning or the 
student experience. In other words, many are performing assessment to gather 
data; few are using the data to a useful end (Banta & Blaich, 2010; Jankowski, 
Timmer, Kinzie, & Kuh, 2018; Kuh & Ikenberry, 2009; Penn, 2011). 
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Another challenge confronting the standard process is the well-known faculty SS 


resistance and critique that perennially accompany assessment initiatives 
(e.g., Penn, 2011; Worthen, 2018; Eisgruber, 2012; Linkon, 2005). At our 
institution, research data from numerous faculty focus groups and individual 
faculty interviews have revealed broad frustration and dissatisfaction with the 
process (Katz, 2010; Weimer, 2013). For many faculty who resist assessment, 
the tasks are seen as a mandatory but perfunctory exercise not connected in any 
meaningful way to teaching and learning. We may regard such resistance as a 
reasonable response to a process that fails to recognize much of what faculty 
value about teaching and learning (Reed, 2016; Pollock, 2016). Moreover, from 
a faculty member’s perspective, articulating student learning outcomes can 
reduce the richness, complexity, and beauty of their discipline to a short list of 
discrete knowledge, skills, and attitudes for students to master (Ewell [2007] 
refers to this as “fractionation”; see also McKernan, 2008; Cain & Hutchings, 
2015; Powell, 2011; Hussey & Smith, 2002). 


From a systems perspective, the strengths and challenges of assessment form a 
correlative relationship: the strengths of standard practice bring with them a 
dark side that yields challenges. To be more specific, in order to conduct learning 
outcomes assessment systematically across schools, programs, and disciplines at 
university, it is necessary to devise a standard process that can be employed by 
faculty assessors who may have little or even no experience with assessment. A 
standard process enables instructors to complete assessments efficiently, with 
the goal of measuring student learning against program goals and student 
learning outcomes so that the resulting data may be used to improve a program's 
curriculum or the student experience. Without a standardized process, systematic 
assessment becomes very difficult to document or use on a large scale. And yet, 
the very system that enables schools to perform assessment at a high level brings 
with it inflexible processes that can result in compromised data, frustrated or 
alienated faculty, and ultimately the subversion of the very purpose of learning 
outcomes assessment. To better understand the complex relationship between 
the strengths of systematization and the challenges endemic to it, let us first turn 


to the systems theory of McDonaldization as developed by sociologist George 
Ritzer. 


Assessment as Rationalized System 


In 1993, Ritzer published the first of several editions of The McDonaldization of 
Society, which uses the business model of the fast food restaurant McDonald’s 
to explore theories of rationalization as they occur in the modern world. Ritzer’s 
project is to lay out his theory of McDonaldization—Ritzer’s neologism describing 
the rationalization of all manner of structures in the late twentieth century—by 
explaining how it functions and giving a broad array of examples to illustrate 


For many faculty who resist 
assessment, the tasks are seen as 
a mandatory but perfunctory 
exercise not connected in any 
meaningful way to teaching 
and learning. 


National Institute for Learning Outcomes Assessment | 5 


the theory as it plays out in modern life. The scope of the examples shows the es 


great extent to which our world has adopted rationalized structures to do its 
business, and the academy is no exception. Ritzer adduces examples such as 
learning management systems (LMSs) and publisher-provided multiple choice 
assessments; student ratings of instruction and citation counting to determine 
a publication’s influence; and Massive Open Online Courses (MOOCs) with 
their automated systems, standardized rubrics, and lectures as examples of 
rationalization in higher education (Ritzer, 2015). To this ever-growing list we 
may add the standard process of learning outcomes assessment, which both 
benefits and suffers from its highly rationalized structure. 


Ritzer builds his theory around four interrelated dimensions—efficiency, 
calculability, predictability, and 
rationalization as a system. Additionally, Ritzer delineates a fifth dimension— 
the irrationality of rationality—to explain the dark side of rationalization, when 


control—which together characterize 


heavily rationalized processes subvert their own purpose. In other words, the 
strengths of rationalization exist in correlation with its weaknesses: the more a 
structure is rationalized, the greater the risk that its rationalization will undercut 
its own purpose. To better understand Ritzer’s theory, let us look at each of its 
dimensions and the strengths they bring to the process of assessment. 


Efficiency entails streamlining and optimizing processes so that the means to the 
end is fast, convenient, and easy. To make the assessment process efficient, we 
standardize the process, making it as straightforward and convenient as possible 
to follow. To keep the process manageable, we limit the expression of learning to 
a handful of goals and student learning outcomes. To make sorting easy, we use 
three or four buckets into which we sort student performance. To avoid extra 
work, we often collect data by counting the number of items students complete 
correctly on multiple choice exams (which are themselves highly rationalized). 
We create or purchase systems—often guided by computer algorithms—to 
manage workflow, organize data storage, and make the work of assessment as 
easy as possible for faculty. 


Calculability is the dimension of rationalization that privileges that which 
can be measured, quantified, or counted with the corollary that quantity 
takes precedence over quality. In assessment, we may well see calculability as 
its premier characteristic among the four dimensions. At most schools, the 
focal point of outcomes assessment remains the numerical data: we look at 
graduation rates, employment figures, graduate salaries; we study test scores; 
and we translate qualitative work into numerical scores with rubrics (which are 
in themselves rationalized devices). We use numbers to tell our story, for we 
know that decision makers in our world are swayed by the allure of numbers. 
‘They are efficient. They appear clear cut. They seem objective, rational, and 
scientific. They help us to reduce the messy, complicated enterprise of teaching 


Rationalization as a system is 
comprised of four interrelated 
dimensions: efficiency, 
calculability, predictability, 
and control. The strengths 

of rationalization exist in 
correlation with its weaknesses: 
the more a structure is 
rationalized, the greater the 
risk that its rationalization will 
undercut its own purpose. 
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and learning into a clear and simple snapshot that purportedly describes the JIS 


extent to which students are learning the goals and outcomes we hope for. 


Translating complicated thinking and operations into numerical scores makes When the four dimensions 


of rationalization work in 
way to proceed without devolving into the chaos of multiple contingencies and ©9601 they enable large 
complications. bureaucratic systems to 


function smoothly and serve 
‘The rationalized dimension of predictability seeks to minimize variability across great numbers of people in 
time and space, emphasizing order, systematization, formalization, routine, 


our work easier and more streamlined (hence tapping into efficiency) when 
making decisions about curricula and programming. Perhaps it is the only 


a fair and straightforward 


and consistency. Given these definitions, the process of assessment is nothing if 
manner. 


not predictable, in particular when computerized systems come into play. The 
digitization of assessment allows for efficient management of work flow and 
provides a digital receptacle not only for data, but also for artifacts of student 
work and other related documents. In order to fit into the system, documents 
must be standardized and thereby made predictable: lists of goals and learning 
outcomes are similar in length, format, and articulation. Student performance 
is sorted into a standard number of buckets, usually three or four. Forms and 
templates standardize the documentation. Rubrics articulate criteria so judgment 
about learning will be consistent and predictable. At our university, assessment 
is so routinized that groups of courses are assessed in every section taught, every 
time it is taught. Predictability enables assessment initiatives to function in 
large settings. Without it, no one would know how to perform assessment from 
course to course and from semester to semester; similarly, no one would know 
how to read and use the data, as it could be reported differently from assessment 
to assessment. 


The fourth dimension of Ritzer’s model of rationalization is control over the 
people involved as a means of avoiding the inefficiencies and unpredictability 
that inevitably accompany autonomous human action. Assessment processes 
depend on linear, sequential work flows that are typically organized and enforced 
by computer systems to ensure the desired process is followed in a timely manner. 
To ensure meaningful data collection, faculty are required to measure specific 
learning against pre-articulated goals and learning outcomes and to follow the 
steps of the process in a certain prescribed order. In academe, where harried 
faculty are burdened with heavy workloads, administrators must often mandate 
the completion of assessment reports, lest the number of assessments completed 
fail to reach a meaningful threshold. Indeed, without the dimension of control, 
exercised by assessment professionals, administrators, and accrediting bodies, we 
might reasonably expect the work of learning outcomes assessment to unravel 
and perhaps even grind to a halt. 


When the four dimensions of rationalization work in concert, they enable large 
bureaucratic systems to function smoothly and serve great numbers of people 
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in a fair and straightforward manner. Examples might include the Internal ees 


Revenue Service, school systems, hospitals, and insurance companies. Without 
rationalized processes, these and other large systems would quickly devolve into 
chaos. However, when the interrelated dimensions of efficiency, calculability, 
predictability, and control are over-emphasized or overused, when they are driven 
beyond their useful limits, when they take precedence over the end they are 
intended to serve, rationalization paradoxically begins to subvert the very goals 
and processes it was meant to streamline and facilitate in the first place. Ritzer 
identifies this paradox as the irrationality of rationality, the term he uses to label 
the fifth dimension of rationalization. And it is through this fifth dimension 
that Ritzer offers his main critique of rationalization. For example, drive-thru 
lanes at fast food restaurant chains are designed as rationalized systems aimed at 
streamlining the acquisition of a meal; yet when used by too many customers at 
once, drive-thru lanes become frustratingly inefficient, fraught with long waits, 
mistaken orders, and short tempers. 


We may similarly consider the irrationality of rationality as it applies to the 
rationalized systems put into place to complete the work of assessment. We 
establish efficient processes to save time, avoid extra work, and complete our 
assessment obligations as expediently as possible. Yet when pushed too far, 
efficiency in assessment practice rubs out the richness of a college education and 
promotes the collection of data without meaning. To ensure program assessment 
is of manageable scope, we limit the number of learning goals and outcomes 
articulated for the program. Yet this limitation for the sake of efficiency also 
artificially reduces the wholeness of a student’s curriculum to a few standardized 
statements of what she or he should know or be able to do at graduation. 
Similarly, to streamline the collection of data, we often choose measurement 
procedures that are easy and undemanding of our time. Yet, not infrequently, 
data gleaned from easy measurement tell us little about the student learning 
that matters most (Muller, 2018; Berg & Seeber, 2016; Champagne, 2011; 
Powell, 2011). Indeed, gathering meaningful data about something as complex 
as the teaching and learning dyad is often expressly inefficient, time-consuming, 
and labor intensive. 


When we consider calculability through the lens of the fifth dimension, similar 
problems begin to emerge. Whether collecting indirect assessment data such as 
graduation rates, employment statistics, graduate salary figures, and grades or 
direct assessment data such as performance on exams, homework, presentations 
or projects, numbers prevail. In their quantifiable simplicity, they can help us 
identify trends and patterns quickly and efficiently, but they can also rub out 
the nuance, variation, details, and context of student learning such that the 
numbers we see actually tell us little about what students have learned well and 
where gaps in their learning persist. The allure of numerical summary can lead 
us to spend valuable time and resources counting and measuring something 


When pushed too far, efficiency 
in assessment practice rubs 

out the richness of a college 
education and promotes the 
collection of data without 
meaning. Indeed, gathering 
meaningful data about 
something as complex as the 
teaching and learning dyad is 
often expressly inefficient, time- 
consuming, and labor intensive. 
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so that we have data to show; yet as David Eubanks points out in his article SS 


lamenting the notorious variance in validity and reliability of assessment data, 
“[I]t is difficult to use assessment results [when] the methods of gathering and 


analyzing data are very poor” (Eubanks, 2017, p. 4). Calculability pushed too Control, when ie ushed too fe ae 


far can lead us to measure what doesn’t really matter at the expense of measuring can yield institutional cultures 


what does so that we have numerical results (Muller, 2018; Berg & Seeber, that collect data that are not 
2016; Powell, 2011). In assessment, the focus on collecting quantifiable data at useful and complete work done 
the expense of meaning often yields spreadsheets of data and plentiful reports with a compliance mentality 
that are nevertheless seldom used to improve student learning. rather than a spirit of asking 


Hutchings’s “genuine questions 


work of faculty, organize data logically, and make assessment reporting uniform about students” in hopes of 
and understandable to all—eliminates the role of professional experience and  #proving student learning or 
judgment in contextualizing and giving meaning to assessment data. When the _— the student experience. 
reporting system is automated, i.e., controlled by a computer algorithm and 


Predictability becomes irrational when standardization—employed to ease the 


therefore standardized, faculty are required to complete assessments in cookie- 
cutter fashion, which can lead to mindless, perfunctory tabulations in order 
to fill in the blanks with data of some sort (Ewell, 2002). The rationalized 
predictability of highly standardized procedures and documentation leads to a 
mentality of compliance and disengagement among faculty: enter the number, 
check the box, and let others decide what it all means. While “good assessment 
begins with real, genuine questions that educators have about their students” (P. 
Hutchings as quoted in Worthen, 2018, para. 12), overly predictable assessment 
systems can have the effect of squelching the “real, genuine questions” arising 
from the professional experience and judgement of educators in favor of 
satisfying system requirements to provide data, which rarely respond to the most 
salient questions about student learning. 


Of the four dimensions of rationality, control seems to morph into the realm of 
the irrational the most readily in the assessment world. In rationalized systems, 
control over human autonomy helps to increase efficiency, ensure predictability, 
and foster calculability by limiting the opportunity for human error, misjudgment, 
and distraction. But when driven too far, control backfires, and this is especially 
the case in the academic context, where the primary actors are accustomed to 
a great deal of autonomy. Most faculty members choose a career in academe 
because they love their subject area, teaching, and the freedom to structure their 
time as they please. For most faculty, mandates from administrators are not 
well received, and for some, they are an affront, purportedly infringing on their 
prized academic freedom (Cain, 2014). When administrators require faculty to 
do the work of assessment, the dimension of contro/ manifests as resentment and 
frustration, which often leads to resistance or counterproductive attitudes, which 
can then even result in sabotage. Some faculty flatly refuse. Some go through 
the motions, collecting and reporting unimportant, sketchy, or even falsified 
data. Some faculty remonstrate loudly against the administration, creating a 
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culture of discontent and frustration. Just as with the other dimensions in the SS aa 


assessment milieu, control, when pushed too far can yield institutional cultures 
that collect data that are not useful and complete work with a compliance 
mentality rather than a spirit of asking Hutchings’s “genuine questions about 
students” in hopes of improving student learning or the student experience 
(Worthen, 2018; Pollock, 2016; Champagne, 2011; Porter, 2012). 


The Faculty Voice 


A careful consideration of the theory of rationalization led us to the conclusion 
that in the abstract, the dimensions of rationalization might bea useful framework 
for understanding assessment in higher education. As we have outlined, the drive 
for efficiency, predictability, calculability, and control, and the accompanying 
consequences suggested by the irrationality of rationality, seemed capable of 
accounting for much of what we as assessment practitioners had experienced 
in our interactions with faculty. Yet we needed to do a more careful analysis of 
actual faculty responses to assessment to determine whether faculty resistance 
could, in fact, be aligned with the dimensions of rationalization. To do so, we 
elicited from faculty at our university their responses to the assessment efforts 
in which they had participated, specifically the curriculum mapping workshops 
offered to help departments complete program review. Although this part of our 
project was not designed to inform yet another article about faculty resistance 
(Gilbert, 2018; Cain & Hutchings, 2015; Pratto, 1996; Ewell, 2002; Linkon, 
2005; Penn, 2011), we began our work by identifying several faculty members 
from the workshops who had expressed reservations or even frustration about 
assessment to participate in semi-structured focus groups and one-on-one 
interviews. Hence, we acknowledge that our recruitment methods may have 
skewed our findings toward more negative responses to assessment, a research 
decision we made in the service of better understanding the relationship 
between faculty resistance and rationalization. 


In both the focus groups and the individual interviews, we asked participants 
to tell us about their experience with the process of assessment, their attitude 
toward the work, whether they had concerns or reservations about assessment 
and pointedly, whether they thought the process of assessment could be reshaped 
to address their reservations and yield actionable improvements to student 
learning. The conversations were recorded (with the participants’ permission, 
and after review of the project by our university's Institutional Review Board), 
and portions of the conversations were transcribed. We organized the faculty 
comments into categories and looked for evidence of alignment with Ritzer’s 
(2015) dimensions of efficiency, calculability, predictability, control, and the 
irrationality of rationality. 


We did a more careful 
analysis of actual faculty 
responses to assessment (e.g.; 
their assessment experiences, 
concerns/reservations, and 
thoughts on whether 
assessment yields actionable 
improvements to student 
learning) to determine 
whether faculty resistance 
could, in fact, be aligned with 
the dimensions of 
rationalization. 
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While our questions addressed assessment of learning outcomes in general, faculty SSS 


in their responses generally focused on the most rationalized aspect of assessment 
at our university, the large-scale assessment of General Education (Gen Ed) 
courses. Our Gen Ed assessment system gathers data each semester from over 
1,000 courses offered in several divisions and schools across the university. 


‘The faculty comments made in our interviews and focus groups illustrate well 
the irrationality of rationality in each of Ritzer’s four primary dimensions. In our 
analysis, we divided faculty comments into three categories: those that addressed 
the efficiency and predictability of the assessment system (grouped together 
because comments touched on both of these dimensions simultaneously); those 
that touched on calculability; and those that addressed control. 


Comments that related to Ritzer’s dimensions of efficiency and predictability 
emphasized the standardization of the assessment process, particularly in the Gen 
Ed context. Faculty recognized the need to standardize the process because of 
the large quantity of data to be collected (all Gen Ed courses at our university 
are assessed in every semester they are offered, resulting in thousands of bits of 
data). But the standardized process, and the frequency of the demand to provide 
assessment data, led faculty to become frustrated with, or disengaged from, the 
process and their own data. One faculty member described her disengagement 
by saying, “It’s easy to translate the process into something mechanical.” When 
the process became mechanical, faculty seemed to want simply to get through it 
rather than engage with their data. Another professor described it this way: 


Whether you take this seriously and try to do an authentic evaluation is 
voluntary. I know faculty members whove told me that what they do is they 
put up their essay prompt, they put up some kind of a rubric, and then they 
put up their grades and they use their grades to determine whether students 
have met their criteria or not. Thats bogus. 


In addition to disengagement, the standardized process led some faculty to express 
frustration with the system’s inflexibility: “The process of aligning grading rubrics 
with outcomes was problematic. It systematized things in a way that left little 
room for changes.” Another professor said, “I got frustrated with the rubrics, 
so I reduced the number of assignments I require so that I wouldn't have to 
align so many rubrics with Gen Ed outcomes.” ‘The faculty disengagement with 
assessment data and results is a problem not only in itself. It also seems to lead 
to skepticism about the whole assessment enterprise, producing a sort of self- 
fulfilling spiral: faculty enter data mechanically, without reflection; that leads 
them to doubt not only their own data but also those of other faculty, which 
in turn causes them to see the entire project as useless. If the process is seen as 
useless, the resulting compliance mentality appears to be a reasonable response. 


Faculty disengagement with 
assessment leads to skepticism, 
producing a self-fulfilling 
spiral: faculty enter data 
mechanically, without 
reflection; that leads them to 
doubt not only their own data 
but also those of other faculty, 
which in turn causes them to 
see the entire project as useless. 
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Faculty comments about the standardization associated with assessment were 
also connected with the idea of lowering the costs of offering courses, an aspect 
of efficiency. One faculty member commented that assessment “creates a level 
of standardization in order to find lower-cost ways to deliver courses.” In this 
instructor's view, the drive for efficiency and predictability would eliminate “the 
specialness of a particular course as it’s taught by a particular instructor.” 


Other comments related to efficiency and predictability emphasized that the 
assessment process failed to meet their needs. One professor noted that the 
efficiency of a standard process actually had the paradoxical effect of wasting 
time: “Some of this seemed like a waste of time because some things didn’t 
apply to us. I wish we could have had a process that was more tailored to 
us.” This is a clear example of the irrationality of rationality, where an efficient 
process subverts its own goal, leading to inefficiency. In faculty comments 
addressing the efficiency and predictability of the assessment system, we see 
faculty interpreting a system that was designed to improve teaching and learning 
instead as a system that degrades it. 


On Ritzer’s theme of calculability, faculty comments again illustrated the 
negative side of this dimension: that numbers fail to capture the nuance and 
complexity of teaching and learning, and that there is a propensity to count 
what is measurable rather than what is meaningful. Numerous faculty expressed 
these ideas in terms like these: 


Students have such different backgrounds and such different reasons for 
taking a course, that assessment is uninformative. I'd have to write a 
biography of each student to tell what actually happened. So I cant take 


the assessment numbers seriously. 


Others expressed similar frustration with the gap between what they saw 
assessment as measuring, and what they saw as meaningful questions: 


Assessment doesnt capture what students can actually do, which includes 
affectively laden stuff, unspoken stuff, nonverbal stuff... Assessment 
doesnt deal well with students who are continually struggling and taking 
steps forward. Does this student show curiosity, engagement, risk-taking— 
thats the kind of question Im interested in. It’s a distinction between 
measuring up to benchmarks and intellectual growth. 


Still others noted a mismatch between their perception of their students’ success 
or failure in an assignment or course and the data they provided. One professor 
noted, “Having taught some of the classes that were evaluated, or worked closely 
with faculty in others, I knew that the results often did not mesh with what I 
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viewed as problems in the course.” Even if accurate and valid data were gathered 
and entered into the system (which may be unlikely, considering Eubanks’ 
[2017] analysis of the problems with assessment methods), faculty described 
issues with the subsequent merging and analysis of the numbers. Referring to 
our university's Gen Ed assessment data entry system, one professor said: 


The bogus part is the last part, where you have to say how many students 
met your criteria, exceeded your criteria, or did not meet your criteria. 
And when I did that, a little bar chart popped up. I suspect that what 
they do is they take all those little bar charts and they amalgamate them. 
Everybody meets the criteria—what criteria? They're all different, in all of 
our different classes. So what we have is “how many apples do we have? I 
dont know, we have a lot of fruit.” Its a statistics fail. 


This quote hints at a concern expressed by other faculty in our interviews: that 
assessment numbers are meaningless because they're combined over so many 
different courses, disciplines, and teaching contexts. This concern and others 
in this category led to two responses from faculty that perfectly illustrate the 
irrationality of rationality. One is adjusting teaching to achieve better numbers, 
a response referred to by Muller in The Tyranny of Metrics (2018, p. 24) as 
“gaming.” That response is shown in this quote: 


[f 20% of the class is with me, and 80% is lost, then I'll delete that 
material next year, so that I can get reasonable numbers. I'll lower the 
requirements in order to meet the standard. 


‘The other response is increasing frustration and resentment of the system and 
cynicism about assessment in general, as seen in these two quotes. 


The most frustrating part is the feeling that you're out there in the 
classroom teaching, wanting your students to learn and doing the best 
you can, meaning well. When someone asks you for your numbers it’s 
frustrating, it’s annoying—it feels like they don’t trust you. But theres no 
way to improve on that. 


Assessment makes us come up with high numbers of students who exceed 
expectations, so we move the bar. So it’s not just a request to assess; it’s a 
request to show that students are meeting or exceeding expectations. 


Itis clear from these responses that the dimension of calculability is interconnected 
with efficiency and predictability, and that in combination, they all illustrate 
the dark side of assessment efforts. In striving for efficiency and predictability, 
the process is standardized rather than being tailored to individual departments’ 
or faculty members’ needs. By eliminating the nuances of teaching and learning 
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in different contexts, the process seems to encourage perfunctory compliance Se 


rather than thoughtful reflection on the data. In striving for calculability, the 
process first asks faculty to submit (into a standardized, mechanical online 
system) data that can seem overly simplistic or even unrepresentative of 
what they think has actually happened in their courses. Then these data are 


When assessment was 
articulated as an issue of 


amalgamated using what may be suspect methods. As a result, the process and control by administrators over 
the results are viewed with frustration and cynicism. faculty, the affective responses 

from faculty were strongly 
Unlike the comments related to efficiency, predictability, and calculability, which negative. 


focused on the assessment process, faculty comments related to Ritzer’s (2015) 
dimension of control focus primarily on the relationship between the faculty 
and university administration. Faculty concerns touching on this dimension 
fell into two categories: resistance to assessment as a top-down mandate from 
administrators, and concerns with academic freedom. Regarding the first of 
these, one faculty member we interviewed noted that she hates the phrase, 
“a culture of assessment.” When asked why, she said, “Again, it has this sort 
of top-down, authoritarian feel to it.” In discussing the curriculum mapping 
workshops, another faculty member noted that “We should be mindful of what 
our courses address. But the top-down mandate felt like the tail wagging the dog 
— it was completely compliance-driven.” The issue of academic freedom was 
raised in terms of concerns over the content of courses and curricula. Almost 
all faculty we interviewed expressed an attitude like this one: “The system isn’t 
useful, and faculty feel like they’re losing control over their curriculum.” ‘This 
idea was often tied to concerns about standardization of courses (an aspect of 
the dimension of efficiency): 


There has been pressure from [the state commission for higher education] 
and the legislature, who want to standardize courses in order to drive 
down costs. But this leads to the faculty losing oversight over the course, 
and losing the specialness of a particular course as it’s taught by a 
particular instructor. 


This quote articulates not only the perceived loss of control over the curriculum 
that results from assessment, but also the concern that assessment will rub out 
the individuality and unique nuances of a particular professor’s teaching of a 
particular course. This is a theme connected to our analysis of efficiency, again 
showing the overlap in these dimensions. When assessment was articulated as 
an issue of control by administrators over faculty, the affective responses were 
strongly negative, as predicted by our earlier theoretical analysis. 


We have a conception of what our jobs are, and now were being told that 
our conception is totally wrong. We're being told we need to rethink our 
notions of what teaching is, and that’s upsetting. 
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I had to collect syllabi from all of our faculty. All the syllabi were 
carefullycrafted, and all the faculty were doing the best they could. And then 
they get asked by someone who knows nothing about the field to provide 
more information. People who don’t know what they're talking about are 
asking us for more information. It’s frustrating as well as insulting. 


A more cynical view of the issue of control was expressed by another faculty 
member: “It’s just a matter of figuring out what the gods of assessment want.” 
Underlying concerns related to Ritzer’s dimension of control is a hint of a 
lack of trust: faculty question the rationale for doing assessment and wonder 
whether administrators have hidden agendas. Several faculty expressed views 
like this: “What is the evidence that assessment makes anything better? How 
would we assess assessment?” In the absence of clear answers to these questions, 
some faculty suggested that assessment may be a politically-driven move by 
administrators to assess faculty, or to make decisions about departments and 
programs (despite repeated assurances by upper level administrators that this is 
not the case). 


In our interviews and focus groups, we asked faculty not only about their 
experiences with assessment and attitudes toward it; we also asked them to 
speculate about how assessment could be reshaped to address their concerns. 
In their responses, faculty had few concrete suggestions. Some suggested 
that a less top-down process might be less frustrating and more useful. The 
faculty member quoted above, who said that she hates the phrase, “a culture of 
assessment,” continued this way: 


But the idea that [the phrase “a culture of assessment”] expresses is an 
important one—one where everybody in their teaching is identifying what 
students are learning, what students are having trouble learning, and 
they're pooling their data not in that quantitative way but in a qualitative 
way. So the individuals are gathering quantitative [data], and it has to 

be quantitative at a certain level because thats a way of summarizing... 
And then you bring that data to conversations within the department. 
And then we have the opportunity to move forward as a collectivity. Thats 
what programmatic assessment is supposed to be, but to a lot of people that 
still feels top-down. 


This quote touches on several themes expressed by other faculty. One is that 
assessment would be better if it focused at a local level within individual 
departments, rather than university-wide: “It has to be indigenous. If it comes 
down from above, it’s the kiss of death.” Another faculty member agreed: “Make 
it so that it’s us telling us what to do, not somebody else telling us what to do.” 
Yet another said, “The best way that administrators could do this is to ask 
faculty to do something. It'll be unique to your department, but do something, 
and document it when youre done.” 


We asked faculty to speculate 
about how assessment could 

be reshaped to address their 
concerns. In their responses, 
some faculty suggested that a 
less top-down process might be 
less frustrating and more useful. 
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Another theme captured in the quote above is the notion that assessment could SSS 


be done in a more holistic, qualitative way. Some said that faculty should be 
allowed “to provide context and additional information in assessment reports,” 
and to “include both qualitative and quantitative information” in their assessment 
reports. They suggested that “faculty should have more conversations about 
these issues without the terminology getting in the way.” One faculty member 
noted that “face-to-face conversations ... would make it feel less compliance- 
driven. It would also get faculty to think beyond their own syllabus.” 


In sum, our analysis of faculty responses to assessment leads us to conclude 
that the theory of rationalization provides a useful framework for understanding 
both the process of assessment and faculty resistance to it. The dimensions of 
efficiency, predictability, calculability, control, and the irrationality of rationality 
(Ritzer, 2015) help identify both the strengths and the problems of assessment as 
it is currently practiced. Rationalization makes large-scale assessment possible, 
but paradoxically, it also eventually undermines its own purpose, as well as 
creating strong faculty resistance to the assessment enterprise. 


The problems engendered by a noble attempt to make assessment possible on 
a mass level must be corrected if we are ever to make assessment relevant to 
the teaching life of our faculty. And since rationalized procedures are so deeply 
embedded in current assessment practice, we believe that the key to re-envisioning 
assessment lies in using an awareness of the dimensions of rationalization in 
order to avoid the most problematic expressions of rationalization. In other 
words, we will apply an inverse of McDonaldization to guide our thought. The 
goal is to develop a new way of doing assessment based on avoiding the problems 
highlighted by rationalization so that the enterprise can more effectively move 
beyond data collection to action. Such an approach could make assessment more 
acceptable to faculty and at the same time focus faculty energy on improving 
student learning and the student experience rather than on the details of data 
collection. 


Foundational Principles of Assessment 2.0 


As we consider alternative, viable assessment methods based on avoiding 
the problems of heavily rationalized systems, we begin by recognizing the 
contributions of other assessment scholars whose ideas have informed our 
own thinking. For example, we can see various themes of Assessment 2.0 in 
Ewell’s (2009; see also Huba & Freed, 2000; Roscoe, 2017) description of 
the “assessment for improvement” paradigm, which emphasizes the formative 
purpose of assessment, and which Ewell contrasts with the assessment for 
accountability paradigm. His assessment for improvement paradigm stresses a 


The problems engendered 

by a noble attempt to make 
assessment possible on a mass 
level must be corrected if we 
are ever to make assessment 
relevant to the teaching life of 
our faculty. 
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focus on assessment methods and questions that are meaningful for the faculty Se 


who will be conducting the assessment. As Ewell notes, “Such questions are 
frequently best framed in terms of particular pedagogical challenges that real 
faculty face in real classrooms” (2009, p. 16). Many other scholars have also : 
emphasized this point (Hutchings, 2010; Maki, 2010; Roscoe, 2017; Linkon, novel ideas or app roaches to 
2005; Steinke and Fitch, 2011; Eubanks, 2017; Reed, 2016). Another theme assessment. Instead, we have 

of Assessment 2.0 emphasized by other scholars is the key role of discussion, attempted to synthesize many 
reflection, and collective meaning-making among departmental faculty as they principles and best practices 
analyze assessment data (Ewell, 2009; Cain & Hutchings, 2015; Roscoe, 2017; from the literature and provide 
Linkon, 2015; Huba and Freed, 2000). Other authors have also highlighted RT ‘framework that 
the benefits of drawing on both the disciplinary expertise and the professional is 

judgment of faculty in designing and conducting assessments (McConnell, helps us understand and AS 
2018; Cain & Hutchings, 2015; Cain, 2014; Hutchings, 2010; Maki, 2010;  #em to design assessment 
Roscoe, 2017; Eubanks, 2017). Finally, the use of flexible assessment methods methods that faculty can own 
that utilize qualitative as well as quantitative data has also been mentioned by and can use to improve student 
numerous authors (Clark and Filinson, 2011; Maki, 2010, McConnell, 2018; 
Cain & Hutchings, 2015; Cain, 2014; Porter, 2012; Ewell, 2009; Linkon, 
2005). It is clear that Assessment 2.0 is not a set of novel ideas or approaches to 
assessment. Instead, we have attempted to synthesize many principles and best 
practices from the literature and provide a theoretical framework that helps us 
understand and use them to design assessment methods that faculty can own 
and can use to improve student learning. 


Assessment 2.0 is not a set of 


learning. 


At the same time, we must be wary of subverting our own project by rationalizing 
a non-rationalized system into a one-size-fits-all solution. In the final section of 
this essay, we will discuss the preliminary results of non-rationalized assessment 
as it is playing out in our context; yet we also caution that our Assessment 2.0 
procedure is not intended prescriptively, but rather as an exemplar of a successful 
process based on foundational principles developed from the theorization we 
have explored above. In other words, it is one example of many possibilities 
for re-envisioning successful assessment procedures that are faculty friendly and 
lead to action. 


We offer the principles below as guidelines for consideration prior to the 
development of Assessment 2.0 variations, which should be tailored to the 
unique goals and needs of each department, program, school, or university. 


Data Quality and Usefulness over Efficiency: While rejecting an 
overemphasis on efficiency does not imply expressly devising an inefficient 
assessment program, it does indicate a shift in priorities, especially in the data 
collection phase. For Assessment 2.0, we believe the process should focus 
foremost on collecting data that will inform us about student learning that 
matters and form the basis for decisions about curricular and instructional 
changes in the students’ best interest. Hence, we believe the process should 
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be less concerned with completing the assessment as quickly and easily as ||. 


possible, and more concerned with finding the best way to discover the gaps 
and fissures in student learning vis-a-vis the learning we care most about. 
Often, for example, instructors identify items on multiple choice exams that 
relate to a particular learning outcome and then complete the assessment 
by categorizing student achievement according to how many of the selected 
items each student answers correctly. This method is fast and easy (efficient) 
but it rarely reveals data detailed enough to inspire confidence among 
faculty and administrators to make curricular changes. A much less efficient 
method, on the other hand, might involve studying student work on one 
multi-step problem that reveals the thought processes and competencies 
indicated by a learning outcome. While this latter assessment procedure 
will surely take more time and effort (not efficient), it will yield data that 
can serve as persuasive evidence of student performance on which curricular 
changes and hopefully improvements can be made. 


Local Applicability over Predictability: Negotiating the inverse of 
predictability can be tricky in an institutional context with many players 
involved in the process. On the one hand, dispensing with predictability 
altogether may translate into confusion, where faculty idiosyncratically 
complete assessments, and data are reported in every which way with no 
common basis or structure. Faculty committees would struggle to interpret 
the results or recognize program-wide patterns. Unpredictable (non- 
standardized) assessment procedures would also emphasize individualized, 
course-level improvements rather than improvement at the program, 
department, or school level because of the individualization of the assessment 
process. On the other hand, requiring faculty to complete assessments 
in lockstep fashion with no option for contextual influences, individual 
findings, or out-of-the-ordinary observations can lead to a compliance 
mentality: check the boxes, fill in the numbers and submit. Hence, rather 
than standardization, we propose a process that asks instructors to focus 
on the production of data that emerge naturally from their experience and 
professional judgment as educators with respect to the student learning in 
their purview. With more latitude for the instructor and less pre-determined 
uniformity, assessment becomes much more faculty friendly and further 
stands to produce data that may have gone unnoticed in a more rationalized 
process. 


Holistic Assessment over Reductive Calculability: In our estimation, one 
of the great problems with current assessment practice is the overreliance 
on quantifying student learning, even when numerical data may be 
inappropriate or lead to inaccuracies or misunderstandings. Perhaps 
because of their simple elegance and ability to summarize information, 
numerical data have an allure that is hard to resist. In standard assessment 


One of the great problems 

with current assessment 
practice is the overreliance 

on quantifying student 
learning, even when numerical 
data may be inappropriate 

or lead to inaccuracies or 
misunderstandings. Assessment 
2.0 asks faculty instead to 

look at student performance 
holistically. 
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practice, rubrics have become de rigueur in the data collection process (see ee 


Jankowski et al., 2018). While they can be extremely useful in marking 


student work and transforming performance into a score, rubrics can also 
ee ci De When faculty have control 
miss key characteristics in student work and focus our attention on aspects 


that don’t matter, even as we fail to notice those that do. Assessment 2.0 and agency over the assessment 
asks faculty instead to look at student performance holistically, asking  P”OCeSS and how the data 

for observations about student learning based on the faculty member’s will be used, their natural 
experience and professional judgment about the student work they have interest in student learning 


reviewed, whether for formative or summative purposes. While we in no Fe aetHivited Wid Essesstient 


becomes an interesting and 
suggest that non-numeric, holistic data may be an important supplement ©" 8*?"S activity instead of a 


that helps us to identify learning gaps and developmental problems that we requirement of compliance. 
might otherwise miss when performance is reduced to a simple numerical 


way wish to suggest that numerical data should be abandoned—they will 
always occupy an important place in the assessment world — we do wish to 


score. 


Subsidiarity over Control: Top-down administrative control over faculty 
in the assessment process is perhaps the greatest threat to the entire enterprise 
because it typically yields resentment, frustration, resistance and sometime 
even sabotage. Part of the problem stems from administrative interference 
with the fabled independent and self-regulative work habits tacitly promised 
to tenured faculty. A second contributing factor, however, is the opacity that 
shrouds how assessment data will be used. When faculty are asked to collect 
data to satisfy administrative bodies with no notion of how those data will 
be used generally or locally to benefit them, resentment and disengagement 
ensue. Hence, we suggest that while top-level administrative support for the 
assessment enterprise is essential, assessment itself should be driven by the 
principle of subsidiarity, where matters are handled at the smallest, lowest, 
or least centralized authority. When groups of faculty who teach closely- 
related courses determine what to assess based on their natural interest and 
curiosity about student learning in their own local contexts, the work of 
assessment takes on direct relevance for the faculty doing the work. When 
faculty have control and agency over the assessment process and how the 
data will be used, their natural interest in student learning is activated and 
assessment becomes an interesting and energizing activity instead for a 
requirement of compliance. 


Actionability over Data Gathering For Its Own Sake: Closely related to 
subsidiarity, but deserving of its own designation is actionability. As noted 
above, all too often assessment work focuses on the procedural steps up to 
and including data collection with little attention to using the data to make 
changes and improvements to curricula, programs, instruction, and other 
dimensions of the student experience. While we may adduce several reasons 
for this lamentable state of affairs, a major contributing factor remains the 
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alienation faculty feel toward the data and their meaning. In order for the | |_| _| 5. 


assessments to matter and be taken seriously, faculty must have agency over 
changing or improving student learning. They must see how the data relate Assessment 2.0 is organic 
to their own experience as instructors and know that they can act to close ‘ 

the gaps and mend the fissures they detect in their own students’ learning. because rather than being 


Accordingly, Assessment 2.0 focuses first on doing assessment that leads to governed by a rationalized, 
action, not simply data collection (Walvoord, 2004). Indeed, so important external system, faculty 
is this principle that we believe assessment should not be done unless there are invited to indicate the 


is real possibility and openness to change. Correlatively, we believe that learnings they hoped to see in 


their students and to use their 
professional experience and 


the most effective assessments begin with faculty curiosity about aspects of 
students learning they suspect are not up to par. When this context obtains, 
assessment can reveal hidden learning problems that instructors can then 


address with interest and enthusiasm. judgment to assess the extent to 
which students mastered those 
Assessment 2.0: An Organic Exemplar learnings. 


Beginning with the foundational principles of Assessment 2.0, we developed and 
piloted an assessment process that eschews the disadvantages of rationalization 
endemic to the traditional assessment process and yet nevertheless gathers 
rich, actionable information about student learning that matters to individual 
instructors and programs. We think of our process as organic because rather than 
being governed by a rationalized, external system, faculty are invited to indicate 
the learnings they hoped to see in their students and to use their professional 
experience and judgment to assess the extent to which students mastered those 
learnings. Instead of tables of quantitative data, individual faculty members 
offer qualitative data that are then reviewed—either by faculty themselves or by 
an assessment expert or consultant—to identify themes and commonalities as 
they naturally emerge from faculty remarks. 


At this juncture, assessment experts may be reacting with skepticism: What 
about program goals and learning outcomes? What about curricular alignment? 
What about looking directly at student work or performance? Just as Ewell 
et al. (2017) express reservation in response to Roscoe's (2017) suggestion 
that improvements to student learning be based on faculty conversation and 
research literature rather than direct (and rationalized) assessment, one might 
reasonably levy the same criticism on Assessment 2.0. That is, “efforts to 
improve that do not start by defining the objectives of instruction are akin to 
sailing without a compass. ... [E]fforts to improve without shared instructional 
goals and associated measurements will not lead to systematic improvement” 
(Ewell et al., 2017, paragraph 11). 


In response to these questions and potential criticisms, we by no means wish 
to suggest that the traditional structures of assessment be abandoned or elided. 
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Each of the well-considered steps of traditional, rationalized assessment is 
indispensable to the overall project of improving curricula and student learning. 


Thus, we do see Assessment 2.0 as a supplement to standard practice, 
providing a fuller, more detailed picture of student learning from which to take 
action and plan improvements. We still begin (see our questions below) by 
asking faculty to tell us their core learning objective(s), but we let them emerge 
naturally, rather than imposing them from without. Interestingly, the 
independently articulated learning objectives collected from faculty aligned 
surprisingly well, both among faculty and with the program's previously 
articulated goals and student learning outcomes. Perhaps we may say that 
Assessment 2.0 navigates with the stars rather than a compass; yet it sails 
with purpose nonetheless. 


For our pilot study, we focused on a single program and selected seven faculty 
members who had considerable experience teaching key courses (in different 
content areas and at different levels) in a program’s curriculum. We sent them 
a questionnaire and asked them to spend no more than 15 minutes responding 
to five simple questions: 


1. What is the most important learning you hoped to see in your course 
this semester? 


2. Based on the student work you've seen, what have been the greatest 
gains in student learning this semester? 


3. Based on the student work you’ve seen, what are the biggest gaps in 
student learning (i.e., what do you wish students had learned better)? 


4, How do you think students might have been better prepared when 
they entered your course—what knowledge, skills, or attitudes did they 
need more of? 


5. Based on your observations, what do students still need to learn or 
improve upon as they exit your class? 


We reviewed and discussed the rich qualitative data provided by all seven 
instructors and grouped their responses into a small number of themes. The 
responses to question 3, about gaps the faculty members had observed in their 
students’ learning, particularly surprised us because of their similarity, even 
across the very different content areas of the courses involved: nearly all the 
instructors noted that their students had limited ability to integrate and apply 
knowledge beyond defined and familiar contexts. Students seemed to memorize 
rather than learn concepts. Faculty frequently offered their responses in this 
form: “Students struggle with X, because they’re doing Y,” which gave a fuller 
view of the learning problem than might have been possible in a traditional 
assessment, because it described what students were doing as well as what they 
werent doing. 
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We presented our findings to the faculty participants and to the head of SS. 


the program, and they found the results significant enough to call a 


meeting of key program faculty to discuss the results and brainstorm ideas ae 
ae a as ; the principles of Assessment 
for improving student learning. After that meeting, a smaller committee 


developed a plan to address the gaps in students’ learning through faculty UTS vee intended to be ; 
development events scheduled for the following semester designed to help restrictive; we are not proposing 
faculty teach skills of application and deep learning. simply another assessment 


method to be rigidly applied 
precisely as we have described 
it, in place of traditional 
methods. 


Our pilot study of organic assessment, while preliminary and small in 
scale, demonstrates the feasibility of conducting assessment in a non- 
rationalized way, by designing the assessment in alignment with the 
principles of Assessment 2.0. Regarding data quality over efficiency, our 
method allowed us to obtain rich qualitative data from which consistent 
themes emerged—themes that touched on ways of thinking at the heart of 
the discipline involved. Yet while avoiding an emphasis on efficiency, our 
methods were paradoxically efficient: the assessment methods in each case 
demanded little faculty time, but yielded significant information. To 
enhance local applicability over predictability, we devised an assessment 
method that left room for faculty to use their professional judgment in 
choosing what specific issues to focus on in their responses. 
Interestingly, while the faculty responded individually to the questions, 
their responses coalesced on a small number of important concerns and 
issues. The method favored holistic assessment over reductionistic quantification 
by asking for qualitative information drawn from faculty members’ judgment 
and experience rather than numbers. The emphasis on subsidiarity over 
control was perhaps the key to this pilot. There was no mandate from 
higher administration to conduct this assessment; participation by the 
faculty was voluntary. Faculty controlled the information they provided 
and the conclusions they drew, and the assessment focused on courses they 
had direct control over. Faculty were not alienated from the information 
they provided. The themes identified from faculty responses were 
significant enough to motivate them to brainstorm possible solutions to 
the learning gaps identified, ensuring that the results were actionable rather 
than data gathering for its own sake. 


Our pilot study demonstrates only one of many possible ways of 
instantiating the principles of Assessment 2.0. The principles themselves are 
not intended to be restrictive; we are not proposing simply another 
assessment method to be rigidly applied precisely as we have described it, in 
place of traditional methods. Rather, we see the principles as generative, 
encouraging the creation of a wide variety of assessment methods to meet the 
needs of faculty in diverse disciplines and teaching contexts. Here are a few 
possibilities. 
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* Faculty from several related departments (e.g., different foreign language 
departments) might identify a teaching challenge or learning gap shared 
across their disciplines, and brainstorm ways to gather evidence to further 
analyze the challenge, or to assess potential strategies to address it. 


¢ Faculty who teach different sections of a Gen Ed course (or related Gen 
Ed courses in a single discipline or department) could decide on a specific 
concept or skill that is a key component of the course(s), and gather 
evidence to determine how well students learn the concept or skill across 
the courses. 


¢ Faculty might combine traditional quantitative assessment with 
Assessment 2.0 principles by meeting at a local level (as a department, 
or as a group of faculty teaching closely related courses) to discuss their 
quantitative data and compare them with more holistic judgments of 
students’ learning to search for commonalities. 


‘These ideas harness the principles of Assessment 2.0 to help faculty find ways 
of making assessment—even large-scale Gen Ed assessment—meaningful at a 
local level. But there are many other possibilities. We call on our colleagues to 
reflect on the methods used in their own contexts in light of our theorization, 
and share other examples of assessment that are consistent with the principles 
of Assessment 2.0. 


Assessment can be designed so that faculty identify aspects of student learning 
that are important to them; keep decisions about assessment methods at a 
local level; rely on their own professional judgment as well as other sources 
of evidence; and arrive at actionable conclusions to address the learning gaps 
they have identified. When we as assessment professionals facilitate this kind of 
organic assessment, we may not only avoid the hostility and resistance associated 
with traditional assessment methods; we may also foster assessment that leads 
to real improvements in student learning. 
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